Determining cardiac arrhythmia from a video of a subject being monitored for cardiac function

ABSTRACT

What is disclosed is a system and method for processing a time-series signal generated by video images captured of a subject of interest in a non-contact, remote sensing environment such that the existence of a cardiac arrhythmia can be determined for that subject. In one embodiment, a time-series signal generated is received. The time-series signal was generated from video images captured of a region of exposed skin where photoplethysmographic (PPG) signals of a subject of interest can be registered. Signal separation is performed on the time-series signal to extract a photoplethysmographic signal for the subject. Peak-to-peak pulse points are detected in the PPG signal using an adaptive threshold technique with successive thresholds being based on variations detected in previous magnitudes of the pulse peaks. The pulse points are then analyzed to obtain peak-to-peak pulse dynamics. The existence of cardiac arrhythmias is determined for the subject based on the pulse dynamics.

TECHNICAL FIELD

The present invention is directed to systems and methods for processinga time-series signal generated by video images captured of a subject ofinterest in a non-contact, remote sensing environment such that theexistence of a cardiac arrhythmia can be determined for that subject.

BACKGROUND

Monitoring cardiac events is of clinical importance in the earlydetection of potentially fatal conditions. Current technologies involvecontact sensors the individual must wear constantly. Such a requirementcan lead to patient discomfort, dependency, loss of dignity, and furthermay fail due to a variety of reasons including refusal to wear themonitoring device. Elderly cardiac patients are even more likely tosuffer from the adverse effects of continued monitoring.

Among many cardiac diseases involving rhythmic disorders, atrialfibrillation (A-fib) represents ⅓ of hospital admissions for cardiacissues. A-fib can cause palpitations, fainting, chest pain, orcongestive heart failure and even stroke. It is one of the most commonsustained arrhythmias. It increases with age and presents with a widespectrum of symptoms and severity. There are over 2 million Americansdiagnosed with A-fib and is most frequent in elderly patients.Unobtrusive, non-contact, imaging based methods are needed formonitoring cardiac patients for A-fib episodes.

Accordingly, what is needed in this art are sophisticated systems andmethods for processing a time-series signal generated by video imagescaptured of a subject of interest in a non-contact, remote sensingenvironment such that the existence of a cardiac arrhythmia can bedetermined for that subject.

INCORPORATED REFERENCES

The following U.S. patents, U.S. patent applications, and Publicationsare incorporated herein in their entirety by reference.

-   “Subcutaneous Vein Pattern Detection Via Multi-Spectral IR Imaging    In An Identity Verification System”, U.S. patent application Ser.    No. 13/087,850, by Xu et al.-   “Deriving Arterial Pulse Transit Time From A Source Video Image”,    U.S. patent application Ser. No. 13/401,286, by Mestha et al.-   “Processing A Video For Vascular Pattern Detection And Cardiac    Function Analysis”, U.S. patent application Ser. No. 13/483,992, by    Mestha et al.-   “Deriving Arterial Pulse Transit Time From A Source Video Image”,    U.S. patent application Ser. No. 13/401,286, by Mestha et al.-   “Estimating Cardiac Pulse Recovery From Multi-Channel Source Data    Via Constrained Source Separation”, U.S. patent application Ser. No.    13/247,683, by Mestha et al.-   “Systems And Methods For Non-Contact Heart Rate Sensing”, U.S.    patent application Ser. No. 13/247,575, by Mestha et al.-   “Filtering Source Video Data Via Independent Component Selection”,    U.S. patent application Ser. No. 13/281,975, by Mestha et al.-   “Removing Environment Factors From Signals Generated From Video    Images Captured For Biomedical Measurements”, U.S. patent    application Ser. No. 13/401,207, by Mestha et al.-   “Detection of Atrial Fibrillation from Non-Episodic ECG Data: A    Review Methods”, S. K. Sahoo et al., 33^(rd) Annual International    Conference of the IEEE EMBS, Boston, Mass. USA, (Aug. 30-Sep. 3,    2011).-   “Three Different Algorithms For Identifying Patients Suffering From    Atrial Fibrillation During Atrial Fibrillation Free Phases Of The    ECG”, by N. Kikillus et al, Computers in Cardiology, 34:801-804,    (2007).-   “Blind Signal Separation: Statistical Principles”, Jean-Francois    Cardoso, Proceedings of the IEEE, Vol. 9, No. 10, pp. 2009-2025,    (October 1998).-   “Independent Component Analysis: Algorithms And Applications”, Aapo    Hyvarinen and Erkki Oja, Neural Networks, 13(4-5), pp. 411-430,    (2000).

BRIEF SUMMARY

What is disclosed is a system and method for processing a time-seriessignal generated by video images captured of a subject of interest in anon-contact, remote sensing environment such that the existence of acardiac arrhythmia can be determined for that subject. Many A-fibdetection algorithms rely on the variability of the RR interval obtainedfrom ECG signals. In the case of A-fib, the chaos and randomness offluctuations of the stroke volumes lead to large fluctuations of thelevels of both the systolic and the diastolic blood pressure. Frequencyand duration of A-fib episodes can also change. Since pulse signals fromvideo images correlate with PPG and ECG peaks, the teachings hereof aredirected to detecting such episodes by measuring peak-to-peak intervalsfrom the blood volume (also called cardiac volumetric) signals extractedfrom time-series signals generated from video images of the subject.These peak-to-peak intervals are associated with consecutive heartbeats. With an implementation of the teachings hereof, cardiacarrhythmias can be discovered in real-time (or processed offline) from avideo captured of the resting cardiac patient. The system and methodsdisclosed herein provide an effective tool for atrial fibrillation studyand cardiac function analysis.

One embodiment of the present method for detecting cardiac arrhythmiafrom signals generated from video images captured of a subject ofinterest being monitored for cardiac function in a non-contact remotesensing environment involves the following. First, a time-series signalgenerated is received. The received time-series signal is generated fromvideo images captured of a region of exposed skin wherephotoplethysmographic (PPG) signals of a subject of interest can beregistered. The video comprise video images captured by at least oneimaging channel that is capable of capturing photoplethysmographicsignals. The video images can be any combination of: NIR images, RGBimages, RGB with NIR images, multispectral images, and hyperspectralvideo images. Signal separation is performed on the received time-seriessignals to extract a photoplethysmographic (PPG) signal for the subject.In various embodiments, performing signal separation on the time-seriessignals comprises performing, using a reference signal, a constrainedsource separation algorithm on the time-series signals to obtain the PPGsignal. The reference signal preferably has a frequency range thatapproximates a frequency range of the subject's cardiac pulse.Peak-to-peak pulse points are detected in the PPG signal using anadaptive threshold technique with successive thresholds being based onvariations detected in previous magnitudes of the pulse peaks. The pulsepoints are then analyzed to obtain peak-to-peak pulse dynamics. Theexistence of a cardiac arrhythmia is determined based on the pulsedynamics. In one embodiment, cardiac arrhythmia is determined using aPoincare diagram of the peak-to-peak pulse dynamics. In anotherembodiment, cardiac arrhythmia is determined based on whether the timeinterval between consecutive peaks in the processed PPG signal isoutside an acceptable limit.

Many features and advantages of the above-described method will becomereadily apparent from the following detailed description andaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other features and advantages of the subject matterdisclosed herein will be made apparent from the following detaileddescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 shows a schematic diagram of normal sinus rhythm for a humanheart as seen on an electrocardiogram (ECG);

FIG. 2 shows the synchronization between the heart rate signalsestimated from the video-based system to a commercial PPG system;

FIG. 3 illustrates one embodiment of an example video camera system foracquiring a video signal of a subject of interest being monitored forcardiac function;

FIG. 4 is a flow diagram which illustrates one embodiment of the presentmethod for detecting cardiac arrhythmia from signals generated fromvideo images captured of the subject of interest of FIG. 3;

FIG. 5 shows a normalized (unfiltered) heart rate signal and anormalized (filtered) heart rate signal filtered using a moving averagefilter with a 40 frame delay;

FIGS. 6A-B show the power spectral densities of the unfiltered andfiltered heart rate signals of FIG. 5, respectively;

FIG. 7 shows heart rate with respect time when calculated using theinverse of the PP(n) and PP_(normalized)(n) intervals;

FIG. 8A shows the Poincare plot PP(n) intervals (i.e., withoutnormalization);

FIG. 8B shows the Poincare plot PP_(normalized)(n) intervals (i.e., withnormalization); and

FIG. 9 illustrates a block diagram of one example processing system 900capable of implementing various aspects of the present method describedwith respect to the flow diagram of FIG. 4.

DETAILED DESCRIPTION

What is disclosed is a system and method for processing a time-seriessignal generated by video images captured of a subject of interest in anon-contact, remote sensing environment such that the existence of acardiac arrhythmia can be determined for that subject.

NON-LIMITING DEFINITIONS

“Cardiac function” refers to the function of the heart and, to a largeextent, to the cardio-vascular system. In most species, the heartcomprises a muscle which repeatedly contracts to pump hemoglobin throughan arterial network. Cardiac function can be impacted by a varietyfactors including age, stress, disease, overall health, and the like.Cardiac function can also be affected by environmental conditions suchas altitude and pressure.

A “subject of interest” refers to a human having a cardiac function.Although the term “human”, “person”, or “patient” may be used throughoutthis text, it should be appreciated that the subject may be somethingother than a human such as, for instance, an animal. Use of “human”,“person” or “patient” is not to be viewed as limiting the appendedclaims strictly to human beings.

A “video” is a sequence of images captured of a subject of interestusing a video camera. The video may also contain other components suchas, audio, time reference signals, noise, and the like. The video mayalso be processed to compensate for motion induced blur, imaging blur,or slow illuminant variation. The video may be processed to enhancecontrast or brightness. Independent component selection can also be usedto emphasize certain content in the video such as, for example, a regioncontaining larger blood vessels. If camera-related noise orenvironmental factors are adversely affecting extraction of cardiacsignals, compensation can be effectuated using the teachings describedin the above-incorporated US patent application entitled: “RemovingEnvironment Factors From Signals Generated From Video Images CapturedFor Biomedical Measurements”, by Mestha et al. Post-compensated videosignals contain decorrelated and noise corrected channels on a per-framebasis.

A “video camera” is a device for acquiring a video. For the purpose ofdetecting cardiac arrhythmias, as disclosed herein, a near infrared(NIR) camera (4-channel or 1-channel) is preferable. Combinations ofvisible and IR with multi/hyperspectral image capture system can also beused. In one embodiment, the video camera comprises a hybrid devicecapable of capturing both color and infrared images. The video cameramay be a multi-spectral or hyperspectral device.

A “video analysis module”, in one embodiment, refers to a hardwaredevice with at least one processor executing machine readable programinstructions for analyzing video images such that cardiac arrhythmiascan be determined in accordance with the teachings hereof. Such a modulemay comprise, in whole or in part, a software application working aloneor in conjunction with one or more hardware resources. Such softwareapplications may be executed by processors on different hardwareplatforms or emulated in a virtual environment. Aspects of the videoanalysis module may leverage off-the-shelf software.

“Cardiac arrhythmia”, also known as cardiac dysrhythmia, means anirregular heartbeat caused by a change in the heart's electricalconduction system.

“Atrial fibrillation” (AF or A-fib), is one of the most common cardiacarrhythmias. In AF, the normal regular electrical impulses generated bythe sinoatrial node are overwhelmed by disorganized electrical impulsesusually originating in the roots of the pulmonary veins, leading toirregular conduction of impulses to the ventricles which generate theheartbeat. FIG. 1 shows a schematic diagram of normal sinus rhythm for ahuman heart as seen on an electrocardiogram (ECG). In atrialfibrillation, the P waves, which represent depolarization of the atria,are absent, with unorganized electrical activity in their place, andirregular R-R intervals due to irregular conduction of impulses to theventricles. Irregular R-R intervals may be difficult to determine if therate is extremely rapid. AF increases the risk of stroke; the degree ofstroke risk can be up to seven times that of the average population,depending on the presence of additional risk factors such as high bloodpressure. It may be identified clinically when taking a pulsemeasurement. The presence of AF can be confirmed with an ECG (or EKG)which demonstrates the absence of P-waves together with an irregularventricular rate. AF may occur in episodes lasting from minutes to days(“paroxysmal”), or be permanent in nature. A number of medicalconditions increase the risk of AF, particularly narrowing of the mitralvalve of the heart (mitral stenosis). Atrial fibrillation may be treatedwith medications to either slow the heart rate to a normal range (“ratecontrol”) or revert the heart rhythm back to normal (“rhythm control”).The evaluation of atrial fibrillation involves diagnosis, determinationof the etiology of the arrhythmia, and classification of the arrhythmia.

A “photoplethysmographic signal”, or simply PPG signal, is a signalwhich correlates to the subject's cardiac pulse pressure wave. In oneembodiment, a region of exposed skin of the subject where such pressurewaves can be registered such as, for example, a neck or chest area, iscaptured by a video camera. The video images are processed to isolate avascular pattern. The vascular network/pattern is identified in thevideo images based on, for example, color, spatial features, materialidentification, and the like. An average of all pixel values in theidentified vascular regions within each image frame of the capturedvideo is computed to obtain a channel average on a per-frame basis. Aglobal channel average is computed, for each channel, by adding thechannel averages across multiple image frames and dividing by the totalnumber of frames. The channel average is subtracted from the globalchannel average and the result divided by a global channel standarddeviation to obtain a zero-mean unit variance time-series signal foreach of the isolated vascular regions. The obtained time-series signalare normalized and filtered to remove undesirable frequencies. Theresulting time-series signals for the vascular regions contain the sumtotal of volumetric pressure changes within those regions. Arterialpulsations comprise a dominant component of these time-series signals.These time-series signals are processed using an independent componentanalysis technique to extract PPG signals. FIG. 2 shows a plot 200 of acardiac signal 201 obtained from a heart rate estimation algorithm asdisclosed in the above-incorporated reference entitled: “EstimatingCardiac Pulse Recovery From Multi-Channel Source Data Via ConstrainedSource Separation”, by Mestha et al., and a PPG signal 202 obtained viaa MP36 Biopac System with a sensor attached to an earlobe of thesubject. Clearly, peak occurrence coincides with signals obtained fromthe PPG system meaning that when the ventricles contract volumetricblood signature of pulsating blood can be seen in the heart rate signalobtained from having processed the video. The present method is based onreflectance-mode photoplethysmography, in which the returned light fromthe skin contains the heart beat signal. The light from the sourceenters into deeper structures of the skin and blood vessels. Althoughthere is some increase in light due to reflected light from moreerythrocytes when blood volume increases, this directly increased lightis negligible when compared to the absorbed light returning from thedeeper tissues. Thus, the negative signal peaks in heart rate signals ofFIG. 2 correspond to increased blood volume which occurs duringventricular contraction. The signals recorded contain blood volumechanges obtained from all blood vessels (e.g., arteries, arterioles,capillaries, venules, and veins) when compared only to electricalsignals obtained using an ECG. These signals are then processed in amanner more fully disclosed herein.

Example Video Capture Device

Reference is now being made to FIG. 3 which illustrates one embodimentof an example video camera system for acquiring a video signal of asubject of interest being monitored for cardiac function in accordancewith the teachings hereof.

Examination room 300 has an example video capture system 302 beingoperated by technician or nurse 303 standing at the bedside 304 ofsubject of interest 305 shown resting his/her head on a pillow whilehis/her body is partially covered by sheet 307. Camera system 302 isrotatably fixed to support arm 308 such that the camera's field of view309 can be directed by nurse 303 onto an area of exposed skin of a chestarea 306 of patient 305 for continuous monitoring of cardiac function.Support arm 308 is on a set of wheels so that the image capture systemcan be moved from bed to bed and room to room. Although patient 305 isshown in a prone position lying in a bed, it should be appreciated thatimages of the subject of interest being monitored for cardiac functioncan be captured while the subject is positioned in other supportingdevices such as, for example, a chair or wheelchair, standing up,including walking or moving. Camera system 302 captures video images ofthe subject of interest to be monitored for cardiac function. Thecaptured video images comprise multi-channel source data such as RGBand/or multi-spectral acquired over time. Camera 302 comprises imagingsensors which may be a single sensor or a sensor array including aplurality of individual or separate sensor units. A central processorintegral to camera 302 and in communication with a memory (not shown)functions to detect changes in the status of sensors and outputting analarm, notice, report, and the like, if a change in any hardware orsoftware of the camera has been detected. Other sensors are capable ofsensing a change of status of patient 305 and issue an alarm ornotification via transmission element 310 to a nurse, doctor, ortechnician in the event that the monitored cardiac function of thepatient falls outside a set of pre-defined parameters.

Antenna 310 is used to communicate the video images to various remotedevices. Transmitter 310 may be a wired (e.g., Ethernet) connectionutilizing an Ethernet network consisting of Ethernet cables and anEthernet hub that is in communication with a network 301. Camera system302 may include both wireless and wired elements and may be connectedvia other means such as coaxial cable, radio frequency, Bluetooth, orany other manner for communicating data. Network 301 receives thetransmitted video signals and wirelessly communicates the received videoimages to various devices such as, for instance, a workstation with adisplay device, for processing. Data is transferred in the form ofsignals which may be, for example, electronic, electromagnetic, optical,light, or other signals. These signals are provided to a communicationsdevice such as a server which transmits and receives data packets bymeans of a wire, cable, fiber optic, phone line, cellular link, RF,satellite, or other medium or communications pathway. Techniques forplacing devices in networked communication are well established. Assuch, a further discussion as to specific networking techniques has beenomitted. Any of the networked devices may include a network interfacecard or network communication system.

Flow Diagram of One Example Embodiment

Reference is now being made to the flow diagram of FIG. 4 whichillustrates one example embodiment of the present method for detectingcardiac arrhythmia from signals generated from video images captured ofa subject of interest being monitored for cardiac function. Flowprocessing begins at step 400 and immediately proceeds to step 402.

At step 402, receive a time-series signal generated from video imagescaptured of a region of exposed skin where photoplethysmographic signalsof a subject of interest can be registered. The video comprises videoimages captured by at least one imaging channel capable of capturing thesubject's photoplethysmographic signals. The time-series signal may beretrieved from a storage device for processing or obtained from a remotedevice over a network.

At step 404, perform signal separation on the time-series signals toextract an estimated photoplethysmographic signal for the subject. BlindSource Separation recovers unobserved signals from a mixed set ofobserved signals without any prior information being known about how thesignals were mixed. Typically, the observed signals are acquired asoutput from sensors where each sensor receives or otherwise detects adifferent combination of source signals. One form of blind sourceseparation is independent component analysis (ICA). ICA is adecomposition technique for uncovering independent source signalcomponents from a set of observations that are composed of linearmixtures of underlying sources, i.e., independent components of theobserved data. Constrained source separation is an independent componentanalysis method for separating time-series signals into additivesub-components using a reference signal as a constraint. In oneembodiment, the reference signal has a frequency range that approximatesa frequency range of the subject's cardiac pulse. Not all constraintscan be used for constrained independent component analysis (cICA)because some constraints infringe classical ICA equivariant properties.Constraints that define or restrict the properties of the independentcomponents should not infringe the independence criteria. Additionalconditions can be incorporated using, for example, sparse decompositionof signals or fourth-order cumulants into the contrast function, to helplocate the global optimum separating the components.

The obtained PPG (heart rate) signal is converted to zero-mean unitvariance. To remove sub-bands in the heart rate signal, a filtering stepis performed in order to improve peak detection accuracy. HR signals canbe filtered using, for example, a moving average filter with a suitablemoving window of size N frames. One example moving average filter isgiven as:

$\begin{matrix}{{y(n)} = {\frac{1}{N}{\sum\limits_{1}^{N}{x\left( {n - i} \right)}}}} & (1)\end{matrix}$

where N is the number of frames in the moving window, x is theunfiltered photoplethysmographic signal, y is the filteredphotoplethysmographic signal, n is the current frame i is the indexdesignating the moving frame. The moving average filter can also providecorrections to missing peaks. Additional corrections may be necessarybased on estimating the average of the amplitudes obtained from previouspeaks. FIG. 5 shows a normalized unfiltered heart rate signal 501 and anormalized heart rate signal 502 filtered with a moving average filterwith a 40 frame delay. Peak points are obtained with a threshold of0.15. The plot of FIG. 5 was obtained from infant videos from a neonatalintensive care unit (ICU). FIG. 6 A&B shows the respective powerspectral densities of the unfiltered and filtered heart rate signals ofFIG. 5. A small phase distortion generated from the moving averagefilter due to the moving window size does not add significantdetrimental effects on the A-fib detection accuracy. Ectopic heartbeats, irregular beats arising with decrease in blood supply to theheart, can give a longer peak to peak (PP) interval. They can either beremoved from the HR signal or replaced with interpolated values (or withvalues determined to be more acceptable).

At step 406, detect peak-to-peak pulse points in thephotoplethysmographic signal. Peak-to peak intervals are relatively easyto detect. Since these peaks coincide with the PPG signal, the count ofthe number of peaks per minute provide an estimate of the heart rate. Toidentify arrhythmia prior to on-set, statistics about peak-to-peakdynamics are obtained such as, for example, the time interval betweentwo consecutive heart beats using, for instance, a Poincare diagram ofpeak-to-peak dynamics to identify A-fib; and excess time taken by thepresent PP interval. A threshold detector is used to determine the peakpulse point. The threshold can be made adaptive in order to avoid false(or missing) pulse peaks if a single threshold is too high. Oneadaptation strategy involves determining a next threshold based on thevariations detected in the previous magnitudes of pulse peaks. The pulsepeaks of FIG. 5 were identified using a peak-to-peak detection algorithmwith a fixed threshold. In other embodiments, peak detection is guidedby an estimated heart rate. To increase the accuracy of peak-to-peakinterval, the time-series signal can also be pre-upsampled to a standardsampling frequency such as, for instance, 256 Hz.

Peak-to-peak intervals are preferably normalized to 60 beats per minuteusing Eq. (2) in order to make it independent from pulse variations.This also helps in those embodiments wherein it is desirable to comparepeak-to-peak intervals across different patients. In one embodiment, thepeak-to-peak intervals are normalized by:

$\begin{matrix}{{{PP}_{normalized}(n)} = {{{PP}(n)} \times \frac{{heart}\mspace{14mu} {rate}_{15\; \sec}}{60}}} & (2)\end{matrix}$

where, PP(n) is the PP interval for the n^(th) cardiac cycle, heartrate_(15sec) is the mean heart rate of the previous 15 seconds, andPP_(normalized)(n) is the normalized PP interval for n^(th) cardiaccycle. FIG. 7 shows the time evolution of the heart rate with respect totime for approximately one minute when calculated by inverting the PPintervals. Clearly, the normalized signal has low pulse variations.

At step 408, analyze the pulse points to obtain peak-to-peak pulsedynamics. FIGS. 8A-B, show the Poincare plots between PP interval of thecurrent cardiac cycle (PP_(n)) to the PP interval of the next cardiaccycle (PP_(n+1)). FIG. 8A shows the Poincare plot of PP(n) intervals(without normalization). FIG. 8B shows the Poincare plotPP_(normalized)(n) intervals (with normalization). These results wereobtained from a video of an infant in a Neonatal ICU. The Poincare plotof a person with a normal sinus rhythm obtained using ECG-based systemwill be close to a straight line with a slope of 45 degree. For anexample of Poincare plot for normal sinus rhythm, the reader isrespectfully directed “Detection of Atrial Fibrillation fromNon-Episodic ECG Data: A Review Methods”, by S. K. Sahoo et al., 33^(rd)Annual International Conference of the IEEE EMBS, Boston, Mass. USA,(Aug. 30-Sep. 3, 2011), and for patients with A-fib to: “Three DifferentAlgorithms For Identifying Patients Suffering From Atrial FibrillationDuring Atrial Fibrillation Free Phases Of The ECG”, by N. Kikillus etal, Computers in Cardiology, 34:801-804, (2007). Poincare plots frompatients with A-fib look very different (not along the 45 degree line).Contours contain various sizes (e.g., circles, triangles, etc.),distributed without much structure. Poincare plots can also be drawn byrotating with respect to x-axis (not shown).

At step 410, determine whether a cardiac arrhythmia exists based uponthe pulse dynamics. A-fib is not usually detected unless specificallylooked for. To identify A-fib in patients, various parameters can beextracted from the Poincare plot such as, for instance, centroid,vertical deviation, horizontal deviation, ratio of vertical andhorizontal standard deviation, ellipse area, correlation coefficient,regression coefficient, and the equation of the regression line. Otherstatistics can be obtained to identify A-fib. Such statistics arediscussed in the above-cited reference: “Three Different Algorithms ForIdentifying Patients Suffering From Atrial Fibrillation During AtrialFibrillation Free Phases Of The ECG”. Statistical data obtained acrossA-fib patients can be compared to those with normal sinus rhythms toidentify the presence of A-fib. A visual inspection of the Poincare plotcaptured over a prolonged duration (at about 1 hour of PP intervals) onA-fib patients can be sufficient to show an indication of A-fib.Additional risk levels can be assigned based on where the points lie onthe Poincare plot. Chronic A-fib is usually preceded by paroxysmalatrial fibrillation (PAT), a premature atrial contraction which triggersa flurry of atrial activity. In some systems, further tuning of theA-fib detection algorithm may be required to estimate the on-set of PAT.Atrial Flutter may also be detected using a high speed IR camera system.

At step 412, communicate the subject's processed PPG signal to a displaydevice. In this particular embodiment, further processing stops. Inanother embodiment, an alarm is initiated which indicates that thesubject's heart arrhythmia is not within acceptable parameters.Initiating an alarm can be, for example, activating a light, making anaudible noise, or otherwise generating a signal which activates a devicewhich, in turn, performs an action or provides a notification. The kindof alarm signal being generated will depend on the particular embodimentwherein the teachings hereof are implemented. In this particularembodiment, once the alarm signal is activated, further processingstops. In another embodiment, processing repeats such that the subject'scardiac function is continuously monitored. The present system can beused in conjunction with other health monitoring equipment or integratedtherewith such that the initiated alarm signal causes these other deviceto perform intended functions.

It should be appreciated that the flow diagrams hereof are illustrative.One or more of the operative steps illustrated in any of the flowdiagrams may be performed in a differing order. Other operations, forexample, may be added, modified, enhanced, condensed, integrated, orconsolidated with the steps thereof. Such variations are intended tofall within the scope of the appended claims. All or portions of theflow diagrams may be implemented partially or fully in hardware inconjunction with machine executable instructions.

Example Functional Block Diagram

Reference is now being made to FIG. 9 which illustrates a block diagramof one example processing system 900 capable of implementing variousaspects of the present method described with respect to the flow diagramof FIG. 4.

The embodiment of FIG. 9, signal processing system 901 receives atime-series signal 902 generated from video images into buffer 903. Thereceived time-series signals have been captured of a region of exposedskin where photoplethysmographic signals of a subject of interest can beregistered. Buffer 903 may be used for queuing information about thereceived signals (or images) such as, for instance, one or more targetregions within the image frames, size of the video, time/dateinformation, and the like. The buffer may be configured to also storedata, mathematical formulas and other representations to facilitateprocessing of the image in accordance with the teachings hereof. Signalseparation module 904 obtains the buffered signal and performs signalseparation on the received time-series signal 902 to extract theestimated photoplethysmographic signal 905 and stores the extractedsignal to storage device 906. Peak detection module 907 receivesphotoplethysmographic signal 905 from signal separation module 904, oralternatively from storage device 908, and detects peak-to-peak pulsepoints in the photoplethysmographic signal, as described with respect tothe flow diagram of FIG. 4, and outputs pulse points. Signal analyzer908 analyzes the pulse points to obtain peak-to-peak pulse dynamics.A-Fib Determinator 909 determines the existence of a cardiac arrhythmiabased upon the pulse dynamics. These results are also displayed ondisplay device 919 in real-time or processed offline. In thisembodiment, if the subject's cardiac arrhythmia parameters are notwithin predetermined levels set by, for example, the subject's cardiacspecialist, then a notification signal is sent using, for example,transmission element 910, which may assume any of a wide variety ofcommunication elements depending on the design of the system wherein theteachings hereof find their intended uses. In another embodiment,cardiac arrhythmia is determined based on whether the time intervalbetween consecutive peaks in the processed PPG signal is outside anacceptable limit. Notification may further involve initiating an audiblesound which provides an indication to the user hereof or specialist thatthe subject's cardiac arrhythmias require attention. Such a notificationmay take the form of a canned audio message or, for instance, a belltone or a sonic alert being activated, or initiating a visible lightwhich provides an indication such as, for instance, a blinking coloredlight. The communicated notification message can be a text, audio,and/or video message. Such embodiments are intended to be encompassedwithin the scope of the appended claims.

Any of the cardiac signals and parameters determined by signalprocessing unit 901 are communicated to workstation 912 andmulti-function print system device 913 for further processing orrendering to hardcopy. The subject's cardiac data may further becommunicated to remote devices over network 911. Many aspects of network911 are commonly known and a further discussion as to the constructionand/or operation of a specific network configuration has been omitted.Suffice it to say, data is transmitted in packets between networkeddevices via a plurality of communication devices and links usingestablished protocols. Data is transferred in the form of signals whichmay be, for example, electronic, electromagnetic, optical, light, orother signals. These signals are provided to a communications devicesuch as a server which transmits and receives data packets by means of awire, cable, fiber optic, phone line, cellular link, RF, satellite, orother medium or communications pathway. Computer workstation 912 isshown comprising a computer case 918 housing a motherboard, CPU, memory,interface, storage device, and a communications link such as a networkcard. The computer workstation is also shown having a display device 919such as a CRT, LCD, or touchscreen display. An alphanumeric keyboard 920and a mouse (not shown) effectuate a user input. In the embodiment ofFIG. 9, computer system 911 implements database 922 wherein variousrecords are stored, manipulated, and retrieved in response to a query.Although the database is shown as an external device, the database maybe internal to computer case 918 mounted on a hard disk housed therein.A record refers to any data structure capable of containing informationwhich can be indexed, stored, searched, and retrieved in response to aquery. Patient information can be stored and/or retrieved to any of therecords in database 922. It should be appreciated that the workstationhas an operating system and other specialized software configured todisplay a variety of numeric values, text, scroll bars, pull-down menuswith user selectable options, and the like, for entering, selecting, ormodifying information displayed on the display device.

Any of the modules and processing units of FIG. 9 are in communicationwith workstation 912 via pathways (not shown) and may further be incommunication with one or more remote devices over network 911. Itshould be appreciated that some or all of the functionality for any ofthe modules of system 901 may be performed, in whole or in part, bycomponents internal to workstation 912 or by a special purpose computersystem. It should also be appreciated that various modules may designateone or more components which may, in turn, comprise software and/orhardware designed to perform the intended function. A plurality ofmodules may collectively perform a single function. Each module may havea specialized processor and memory capable of retrieving and executingmachine readable program instructions. A module may comprise a singlepiece of hardware such as an ASIC, electronic circuit, or specialpurpose processor. A plurality of modules may be executed by either asingle special purpose computer system or a plurality of special purposecomputer systems in parallel. Connections between modules include bothphysical and logical connections. Modules may further include one ormore software/hardware modules which may further comprise an operatingsystem, drivers, device controllers, and other apparatuses some or allof which may be connected via a network. It is also contemplated thatone or more aspects of the present method may be implemented on adedicated computer system and may also be practiced in distributedcomputing environments where tasks are performed by remote devices thatare linked through network 911.

It will be appreciated that the above-disclosed and other features andfunctions, or alternatives thereof, may be desirably combined into manyother different systems or applications. Various presently unforeseen orunanticipated alternatives, modifications, variations, or improvementstherein may become apparent and/or subsequently made by those skilled inthe art which are also intended to be encompassed by the followingclaims. Accordingly, the embodiments set forth above are considered tobe illustrative and not limiting. Various changes to the above-describedembodiments may be made without departing from the spirit and scope ofthe invention. The teachings hereof can be implemented in hardware orsoftware using any known or later developed systems, structures,devices, and/or software by those skilled in the applicable art withoutundue experimentation from the functional description provided hereinwith a general knowledge of the relevant arts. Moreover, the methodshereof can be implemented as a routine embedded on a personal computeror as a resource residing on a server or workstation, such as a routineembedded in a plug-in, a driver, or the like. The methods providedherein can also be implemented by physical incorporation into an imageprocessing or color management system. Furthermore, the teachings hereofmay be partially or fully implemented in software using object orobject-oriented software development environments that provide portablesource code that can be used on a variety of computer, workstation,server, network, or other hardware platforms. One or more of thecapabilities hereof can be emulated in a virtual environment as providedby an operating system, specialized programs or leverage off-the-shelfcomputer graphics software such as that in Windows, Java, or from aserver or hardware accelerator or other image processing devices.

One or more aspects of the methods described herein are intended to beincorporated in an article of manufacture, including one or morecomputer program products, having computer usable or machine readablemedia. The article of manufacture may be included on at least onestorage device readable by a machine architecture embodying executableprogram instructions capable of performing the methodology describedherein. The article of manufacture may be included as part of anoperating system, a plug-in, or may be shipped, sold, leased, orotherwise provided separately either alone or as part of an add-on,update, upgrade, or product suite. It will be appreciated that variousof the above-disclosed and other features and functions, or alternativesthereof, may be combined into other systems or applications. Variouspresently unforeseen or unanticipated alternatives, modifications,variations, or improvements therein may become apparent and/orsubsequently made by those skilled in the art which are also intended tobe encompassed by the following claims. Accordingly, the embodiments setforth above are considered to be illustrative and not limiting. Variouschanges to the above-described embodiments may be made without departingfrom the spirit and scope of the invention. The teachings of any printedpublications including patents and patent applications, are eachseparately hereby incorporated by reference in their entirety.

What is claimed is:
 1. A method for detecting cardiac arrhythmia fromsignals generated from video images captured of a subject of interestbeing monitored for cardiac function in a non-contact remote sensingenvironment, the method comprising: receiving a time-series signalgenerated from video images captured of a region of exposed skin where aphotoplethysmographic (PPG) signal of a subject of interest can beregistered, said video comprising video images captured by at least oneimaging channel capturing PPG signals; performing signal separation onsaid time-series signal to extract a PPG signal for said subject;detecting peak-to-peak pulse points in said PPG signal; analyzing saidpulse points to obtain peak-to-peak pulse dynamics; and determiningwhether a cardiac arrhythmia exists based on said pulse dynamics.
 2. Themethod of claim 1, wherein said video images comprise any combinationof: NIR images, RGB images, RGB with NIR images, multispectral images,and hyperspectral video images.
 3. The method of claim 1, wherein, inadvance of obtaining said time-series signal, pre-processing said videoto compensate for any of: a motion induced blur, an imaging blur, andslow illuminant variation.
 4. The method of claim 1, wherein performingsignal separation on said time-series signal comprises performing, usinga reference signal, a constrained source separation algorithm on saidtime-series signal to obtain said PPG signal.
 5. The method of claim 4,wherein said reference signal has a frequency range that approximates afrequency range of said subject's cardiac pulse.
 6. The method of claim1, wherein said peak-to-peak pulse points are detected in said PPGsignal using an adaptive threshold technique with successive thresholdsbeing based on variations detected in previous magnitudes of said pulsepeaks.
 7. The method of claim 1, further comprising pre-processing saidtime-series signal by upsampling said signal to a standard samplingfrequency in order to enhance the accuracy of detecting saidpeak-to-peak pulse points.
 8. The method of claim 1, wherein saidcardiac arrhythmia is determined using a Poincare diagram of saidpeak-to-peak pulse dynamics, said Poincare diagram showing arelationship between consecutive beats.
 9. The method of claim 1,wherein, in advance of detecting said peak-to-peak pulse points, furthercomprising filtering said signal using a moving average comprising:${y(n)} = {\frac{1}{N}{\sum\limits_{1}^{N}{x\left( {n - i} \right)}}}$where N is the number of frames in the moving window of said video, x isthe unfiltered PPG signal, y is the filtered PPG signal, n is thecurrent frame and i is an index designating the moving frame.
 10. Themethod of claim 1, further comprising normalizing said detectedpeak-to-peak pulse points to a frequency of 60 bpm to reduce pulsevariations in said PPG signal.
 11. The method of claim 1, whereindetermining a cardiac arrhythmia comprises determining whether a timeinterval between consecutive peaks in said PPG signal is outside anacceptable limit for said subject.
 12. The method of claim 1, whereinsaid time-series signal comprises one of: stored values, and valuesgenerated from a streaming video.
 13. The method of claim 1, furthercomprising comparing said subject's peak-to-peak pulse dynamics acrossdifferent patients.
 14. The method of claim 1, further comprisingcommunicating said subject's peak-to-peak pulse dynamics to a displaydevice.
 15. A system for detecting cardiac arrhythmia from signalsgenerated from video images captured of a subject of interest beingmonitored for cardiac function in a non-contact remote sensingenvironment, the system comprising: a display device; and a processor incommunication with a memory, said processor executing machine readableinstructions for performing: receiving a time-series signal generatedfrom video images captured of a region of exposed skin where aphotoplethysmographic (PPG) signal of a subject of interest can beregistered, said video comprising video images captured by at least oneimaging channel capturing PPG signals; performing signal separation onsaid time-series signal to extract a PPG signal for said subject;detecting peak-to-peak pulse points in said PPG signal; analyzing saidpulse points to obtain peak-to-peak pulse dynamics; determining whethera cardiac arrhythmia exists based on said pulse dynamics; andcommunicating said pulse dynamics to said display device.
 16. The systemof claim 15, wherein said video images comprise any combination of: NIRimages, RGB images, RGB with NIR images, multispectral images, andhyperspectral video images.
 17. The system of claim 15, wherein, inadvance of obtaining said time-series signal, pre-processing said videoto compensate for any of: a motion induced blur, an imaging blur, andslow illuminant variation.
 18. The system of claim 15, whereinperforming signal separation on said time-series signal comprisesperforming, using a reference signal, a constrained source separationalgorithm on said time-series signal to obtain said PPG signal.
 19. Thesystem of claim 18, wherein said reference signal has a frequency rangethat approximates a frequency range of said subject's cardiac pulse. 20.The system of claim 15, wherein said peak-to-peak pulse points aredetected in said PPG signal using an adaptive threshold technique withsuccessive thresholds being based on variations detected in previousmagnitudes of said pulse peaks.
 21. The system of claim 15, furthercomprising pre-processing said time-series signal by upsampling saidsignal to a standard sampling frequency in order to enhance the accuracyof detecting said peak-to-peak pulse points.
 22. The system of claim 15,wherein said cardiac arrhythmia is determined using a Poincare diagramof said peak-to-peak pulse dynamics, said Poincare diagram showing arelationship between consecutive beats.
 23. The system of claim 15,wherein, in advance of detecting said peak-to-peak pulse points, furthercomprising filtering said signal using a moving average comprising:${y(n)} = {\frac{1}{N}{\sum\limits_{1}^{N}{x\left( {n - i} \right)}}}$where N is the number of frames in the moving window of said video, x isthe unfiltered PPG signal, y is the filtered PPG signal, n is thecurrent frame and i is an index designating the moving frame.
 24. Thesystem of claim 15, further comprising normalizing said detectedpeak-to-peak pulse points to a frequency of 60 bpm to reduce pulsevariations in said PPG signal.
 25. The system of claim 15, whereindetermining a cardiac arrhythmia comprises determining whether a timeinterval between consecutive peaks in said PPG signal is outside anacceptable limit for said subject.